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Abstract 



The minute-to-minute and day-by-day use of assessment for learning holds great potential to 
change the trajectory of student learning in U.S. classrooms. But without effective and scalable 
systems of professional development that actually lead teachers to adopt these practices, the 
utility and impact of assessment for learning will be quite limited. Drawing on learning theory, 
expertise research, and research on effective professional development, this paper presents the 
theoretical and empirical basis behind an evolving program of professional development that 
employs school-embedded teacher learning communities as a central component, called Keeping 
Learning on Track® . Implementations of the Keeping Learning on Track program in two districts 
and a consortium of districts are described in detail. 
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Introduction 

Evidence of the effectiveness of assessment for learning (sometimes referred to as 
formative assessment) is convincing and steadily accumulating(for reviews of research, see 
Black & Wiliam, 1998; Borko, 2004; Brookhart, 2005). Furthermore, many assessment for 
learning techniques involve relatively small changes in teaching practice that can leverage large 
improvements in student learning (Ciofalo & Leahy, 2006; Leahy & Wiliam, 2006; Lyon, Wylie, 
& Goe, 2006; Wylie & Ciofalo, 2006). The teaching techniques that enact the five strategies and 
big idea of assessment for learning (Leahy, Lyon, Thompson, & Wiliam, 2005) are decidedly 
low tech, low cost, and within the capabilities of individual teachers to implement. In this way, 
they differ dramatically from large-scale interventions like class size reduction or curriculum 
overhauls, which can be quite expensive and difficult to implement because they require school 
or system-level changes. Taken together, these points suggest that assessment for learning holds 
great potential to change the trajectory of student learning in U.S. classrooms. 

Given the reported effectiveness, low cost, and relative ease of implementation of 
assessment for learning, it is fair to ask why it is not already more widely understood and 
practiced. One reason is that using this approach thrusts teachers and students into new, 
sometimes challenging roles that require changes to the everyday routines of most classrooms 
(Brousseau, 1997). Other substantial cognitive and behavioral shifts are required of teachers as 
well, such as the shift from an emphasis on teaching to an emphasis on learning, a shift in 
assessment mindset from quality control to quality assurance, and a blurring of the boundaries 
between instruction and assessment (Leahy et ah, 2005). 

For teachers to wholeheartedly take on the new roles and new paradigms that assessment 
for learning requires of them, they need more than just a quick exposure to its principles and 
methods. Many teachers have a great deal of the required knowledge and skills to understand and 
implement assessment for learning strategies once they are exposed to these ideas, but they need 
sustained opportunities to consciously develop, practice, reflect upon, and refine this skill set so 
that it works within the context of their own classrooms. Mandated state standards, testing, 
pacing guides, and scripted curricula have left many teachers feeling divorced from goal setting 
and assessment, core practices in assessment for learning. These teachers need more: In addition 
to opportunities for learning, practice, and reflection, they also need experiences that explicitly 
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counteract the isolation, frustration, and de-professionalization that have occurred in many 
school faculties. 

Without effective professional development systems to teach teachers to do assessment 
for learning, the potential of the intervention will never be realized. By effective, we mean that 
the professional development leads to observable, measurable improvements in teaching 
practice, a requisite step toward improving student learning. The sad truth is that most 
professional development going on in U.S. schools is not effective, by this definition. The 
challenge is to develop models of professional development and scalable systems of delivery that 
faithfully disseminate the content of assessment for learning, while also providing sustained, 
meaningful assistance to teachers who are attempting to replace long-standing (perhaps 
unconscious) practices with more effective ones. 

In this paper, we describe how researchers at ETS applied the general notion of school- 
embedded teacher learning communities as a central component of an assessment for learning 
professional development program called Keeping Learning on Track®. As part of the 
development process for Keeping Learning on Track, we piloted a range of delivery models 
across numerous schools and districts. We focus here on three delivery models employed in two 
districts and a consortium of districts. We compare and contrast the delivery models in light of 
each model’s impact on teaching practices and scalability. The data we consider span 2 years of 
our development process, with greater emphasis on our experiences in the current school year 
(2005-2006). Because the data are still coming in, we are able to report only preliminary 
findings at this time. However, these findings are sufficient to guide us in developing a tentative 
framework for the most effective methods for establishing, maintaining, and promulgating 
teacher learning communities as a vehicle for spreading and deepening teachers’ work in 
assessment for learning. 



Why Teacher Learning Communities? 

The Need to Attend to Both Content and Process 

A growing body of research shows that to be successful, teacher professional 
development needs to address both content^ and process aspects of teacher growth (Reeves, 
McCall, & MacGilchrist, 2001; Wilson & Berne, 1999). The content of Keeping Learning on 
Track is focused equally on two ideals. First, it seeks to change the day-to-day instructional 
practices of teachers to include extensive use of five key strategies of assessment for learning: 
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1. Engineering effective classroom discussions, questions, and learning tasks that elicit 
evidence of learning. 

2. Clarifying and sharing learning intentions and criteria for success. 

3. Providing feedback that moves learners forward. 

4. Activating students as the owners of their own learning. 

5. Activating students as instructional resources for one another. 

Second, it incorporates one big idea that unifies and motivates all these strategies — that 
information about student learning should be used to adapt instruction in real time to meet the 
immediate learning needs of students. Keeping Learning on Track is a direct descendent of the 
British research on assessment for learning conducted by Black and Wiliam (1998) and Black, 
Harrison, Lee, Marshall, and Dylan (2002, 2003). The strong framing provided by the five 
strategies and one big idea, plus the various techniques that have been developed for their 
implementation, provide sturdy content for professional development. See Leahy et al. (2005) for 
more on the theory and practical application of the five strategies and one big idea of assessment 
for learning in U.S. schools. 

The primary process by which Keeping Learning on Track attempts to effect these 
changes in teachers’ practice is via school-embedded teacher learning communities, which have 
the potential to provide teachers with the information and support they need to develop their 
practice in deep and lasting ways, and are designed to build school capacity to support individual 
and institutional change over time. 

Teacher Expertise 

Teacher learning communities embody critical process elements needed for professional 
development to result in actual changes in teacher practice. Specifically, effective professional 
development is related to the local circumstances in which the teachers operate (Cobb, McClain, 
Lamberg, & Dean, 2003), takes place over a period of time rather than being in the form of 1-day 
workshops (Cohen & Hill, 1998), and involves teachers in active, collective participation (Caret, 
Porter, Desimone, Birman, & Yoon, 2001; Ingvarson, Meiers, & Beavis, 2005). 

Of course, many professional development structures would be consistent with this 
research base, but we believe that teacher learning communities, as advocated in the Standards 
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for Staff Development under the name professional learning communities (National Staff 
Development Council [NSDC, 2001]), provide the most appropriate vehicle for helping teachers 
become skilled practitioners of assessment for learning. 

We were initially propelled toward this conclusion by personal experience with an 
earlier, failed model of assessment for learning professional development, which did not provide 
adequate, contextualized, sustained support for teacher change. As we internalized the lessons of 
that earlier effort, we looked closely at the growing evidence base on how to build and sustain 
teacher learning communities (Borko, 1997, 2004; Borko, Mayfield, Marion, Flexer, & Gumbo, 
1997; Elmore, 2002; Garmston & Wellman, 1999; Kazemi & Franke, 2003; McLaughlin & 
Talbert, 1993; Putnam & Borko, 2000); Sandoval, Deneroff, & Franke, 2002). We noted a 
pattern in this literature: If teachers’ practices are recurrent, central, and entrenched within 
everyday teaching and school culture, then teachers will need sustained support to change those 
practices. Not only must the support be sustained over time (a year or longer in many of the 
studies cited above), that support must also embed teachers’ learning within the realities of day- 
to-day teaching in their own schools and classrooms, allowing for repeated cycles of learning, 
practice, reflection, and adjustment within their daily context. 

To some extent, these cycles map onto the cyclical depiction of knowledge creation and 
knowledge transmission of Nonaka and Takeuchi (1995), wherein the tacit knowledge of a 
person or group is turned into explicit knowledge so that it can be taught to another person or 
group. Until the new knowledge is practiced and made operational (through a process labeled 
learning by doing, in Nonaka and Takeuchi’ s framework), the knowledge remains explicit. Only 
through sufficient learning by doing can that knowledge be combined with existing knowledge 
structures, internalized, and made accessible and useful in relatively seamless ways (essentially 
making the knowledge tacit again). Nonaka and Takeuchi’ s framework seems particularly apt in 
this application because of its treatment of learning as situated in a social milieu, which is 
certainly characteristic of teacher learning. 

The seamless, transparent, and highly accessible quality of internalized tacit knowledge is 
one of the distinguishing features of expertise, in any field, and assessment for learning is no 
different. An expert in assessment for learning is able to rapidly note essential details of the 
complex social and psychological situation of a lesson (especially the state of student learning), 
while disregarding distracting, yet nonessential details. He or she is then able to swiftly compare 
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that situation with the intended goals for the lesson, the knowledge of the content being taught, 
the developmental knowledge of students in general and these students in particular, and other 
relevant schema. Guided by the results of these comparisons, the expert then selects his or her 
next instructional moves from a wide array of options — most well-rehearsed, some less familiar, 
and some invented on the spot — so that these next steps address the students’ immediate learning 
needs in real time. 

Such expertise is certainly marked by the speed of cognition, but more is needed than 
speed alone. Expert teachers don’t just think faster than nonexpert teachers; they think and 
behave in qualitatively different ways. This behavior has been borne out in the work of Berliner 
(1994), who documented eight ways that expert teachers function like experts in other fields. For 
example, Berliner notes that expert teachers perceive meaningful patterns in the domain of their 
expertise where nonexperts cannot. 

The story of how Berliner came to understand this particular feature of expert teaching is 
instructive and directly related to the need for teachers to practice and reflect upon teaching in 
real contexts. In the early 1990s, he produced a series of videotapes depicting common teaching 
problems in staged classroom settings. When he showed these tapes to novice teachers, 
experienced-but-nonexpert teachers, and expert teachers, he expected the experts to be able to 
describe the videotaped interactions in rich detail and provide plausible, nuanced solutions to the 
problems revealed. Instead, they found that the experts were completely stymied by the 
videotapes, whereas the novices and other nonexperts were able to converse at length about what 
they had seen (though not necessarily cogently or plausibly). Through conversations with the 
expert teachers, he discovered that the staged depictions felt realistic only to nonexperts. Subtle 
but essential details of real classroom life were either absent from the staged depictions or out of 
sync. The nonexperts did not miss or notice these. Without these subtle details in their proper 
place, however, the experts were thrown off in their search for meaningful patterns; they couldn’t 
even begin to make sense of what they were seeing because it did not map onto their relatively 
dense knowledge webs concerning what goes on in teaching and learning in a real classroom. 
When the staged videos were replaced with videos filmed in real classrooms, the experts were 
easily able to respond with detailed, nuanced, cogent, and plausible descriptions and 
prescriptions — even though the technical quality of these spontaneous videos sometimes made it 
difficult to hear and see all the relevant action. 
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This same kind of observation and pattern-matching is an integral part of the teacher’s 
role in assessment for learning, as it is for most complex teaching behaviors. Learning to do 
assessment for learning requires the development of expertise, not the rote application of 
declarative or procedural knowledge. For this reason, we sought a learning vehicle that would 
support the kind of socially supported knowledge creation and transfer described by Nonaka and 
Takeuchi (1995) to feed the development of teacher expertise in general and expertise in 
assessment for learning in particular. Teacher learning communities seemed to have potential to 
accomplish this goal and to represent a learning modality that could be scaled to reach many, 
many teachers in all kinds of schools. 

Are Teacher Learning Communities the Only Way to Go for Professional Development in 
Assessment for Learning? 

Though the idea of teacher learning communities is usually warmly greeted by teachers — 
who generally wish for more collegiality in their professional lives — we did not elect to employ 
them just to be nice to teachers; we chose this vehicle because it is the only one we’ve found that 
works to change teachers’ practices. We arrived at teacher learning communities as a central 
(though not exclusive) learning vehicle for Keeping Learning on Track through a combination of 
experience and our theory-based search for a way to provide contextually relevant, socially 
mediated learning opportunities to teachers. 

We were also attracted to teacher learning communities for Keeping Learning on Track 
because the grassroots character of learning communities lends itself to scaling up the 
intervention. As Black et al. (2002) noted, few teachers make use of assessment for learning in 
day-to-day teaching. Once the notion that it would be good for teachers to do more of this kind 
of teaching is accepted, then the need for professional development is tremendous and the issue 
of scalability rises to crucial importance. Devising a program of professional development that 
works effectively when delivered by its original developers and their hand-picked expert trainers 
is not enough. Where would we find (and how would we pay) the army of experts needed in the 
100,000-plus U.S. schools that could benefit from assessment for learning? There simply are not 
enough qualified coaches and workshop leaders to be found, and the mechanisms for 
disseminating learning through such top-down models are dauntingly complex and expensive. 

Serious challenges will be involved in bringing teacher learning communities to scale 
with fidelity to the original assessment for learning content, given that we must assume that no 
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experts (at least not at the outset) will be part of any particular learning community. We contend 
that this design issue must be faced squarely, and in this paper we will discuss how we did so in 
the various delivery models with varying degrees of success. 

Based on the organizational change literature (Hendry, 1996; Schein, 1996; Stace, 1996) 
and research on school reform (Cobb et al., 2003; Coburn, 2003; Joyce, 2004), 
institutionalization of the intervention within the structures and cultures of schools will greatly 
strengthen the potential for assessment for learning to lead to improved student achievement on a 
wide scale. But until institutionalization occurs, a bootstrapping problem must be addressed. It 
appears that teacher learning communities might be a feasible way to both home-grow 
assessment for learning expertise and foster school cultures that are more amenable to their 
institutionalization. 

Teacher learning communities seem to be particularly functional vehicles to support 
teacher learning about assessment for learning in several ways. First, the practice of assessment 
for learning depends upon a high level of professional judgment on the part of teachers, so it is 
consistent to build professional development around a teacher-as-local-expert model. Second, 
school-embedded teacher learning communities are sustained over time, allowing change to 
occur developmentally, which in turn increases the likelihood of the change sticking at both the 
individual and school level. Third, teacher learning communities are a nonthreatening venue 
allowing teachers to notice weaknesses in their content knowledge and to get help with these 
deficiencies from peers. For example, in discussing an assessment for learning practice that 
revolves around specific content (e.g., by examining student work that reveals student 
misconceptions), teachers often confront gaps in their own subject matter knowledge, which can 
be remedied in conversations with their colleagues. 

In a related vein, teacher learning communities redress a fundamental limitation of 
assessment for learning: its (perhaps paradoxical) generality and specificity. The five assessment 
for learning strategies are quite general; we have seen each of them in use at every level from 
pre-K to graduate-level studies, and across all subjects. Yet implementing them effectively 
makes significant demands on subject matter knowledge. Teachers need strong content 
knowledge to ask good questions, to interpret the responses of their students, to provide 
appropriate feedback that focuses on what to do to improve, and to adjust their teaching on the 
fly based on the information they are gathering about their students’ understanding of the 



7 




content. A less obvious need for subject matter knowledge is that teachers need a good overview 
of the subject matter in order to be clear about the big ideas in a particular domain, so that these 
ideals are given greater emphasis. Teacher learning communities provide a forum for supporting 
teachers in converting the broad assessment for learning strategies into lived practices within 
their specific subjects and classrooms. 

Finally, teacher learning communities are embedded in the day-to-day realities of 
teachers’ classrooms and schools, and as such provide a time and place where teachers can hear 
real-life stories from colleagues that show the benefits of adopting these techniques in situations 
similar to their own. Without that kind of local reassurance, there is little chance teachers will 
risk upsetting the prevailing classroom contract (Brousseau, 1997); while limiting, the old 
contract at least allows teachers to maintain some form of order and matches the expectations of 
most principals and colleagues. As teachers adjust their practice, they are risking both disorder 
and less-than-accomplished performance on the part of their students and themselves. Being a 
member of a community of teacher-learners engaged together in a change process provides the 
support teachers need to take such risks. 

Who Should Lead Teacher Learning Communities? 

Expertise in assessment for learning is developed when teachers learn by doing. It can 
only be developed in those who have ample opportunity for practice, reflection, and 
adjustment — teachers. Just as chess masters need to play a lot of chess to become expert at chess, 
teachers also need to practice assessment for learning to become expert at it. 

Coaching by curriculum coaches or building principals is a common model for school- 
based professional development. However, its utility for advancing assessment for learning is 
limited unless the coach has previously developed expertise in assessment for learning through 
use and refinement in his or her own classroom. We cannot assume that any majority of coaches 
and principals developed this expertise before they left the classroom. This is not to say that 
coaches and principals can play no useful role in supporting teacher learning communities 
focused on assessment for learning, but we do think they should refrain from holding themselves 
up as experts, unless they have walked the walk. 

While we have not had a chance to make a formal study of learning community 
leadership, a cursory review of the results we have seen to date suggests that teachers themselves 
can provide effective leadership for their peers. Because they are going through the same 
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learning and change process, they have essential insights into the pace of change, the kinds of 
dilemmas faced, and the types of support that make sense, all within the context of the 
classroom, not just the building. When we place teachers in the leadership role, we caution them 
not to assume any extra expertise just because they are the facilitator and advocate for the teacher 
learning community, which is often a relief to them because they do not want to hold themselves 
above their peers or feel pressured to act like they know more than they do. They want the 
breathing room to ask for help in the places they are struggling. 

Are Teacher Learning Communities the Only Way to Go for Professional Development? 

Teacher learning communities are certainly catching on, with federal and state education 
policy now moving to acknowledge that these kinds of embedded, teacher-driven, drip feed 
approaches can be an effective way to shift teacher practice. (See, for example. Division of 
Abbott Implementation, 2005; Librera, 2004; North Carolina Department of Public Instruction, 
2005; U. S. Department of Education, 2005). We note, however, that implementing teacher 
learning communities consistently and effectively is not as simple as changing federal and state 
regulations and funding frameworks for professional development. 

Significant structural barriers to teacher learning communities exist in many schools, 
including daily and weekly schedules that provide little or no time with colleagues during the 
normal school day, personnel policies and practices that do not recognize or value teacher 
expertise, local bargaining agreements that discourage teachers from meeting outside scheduled 
hours, inadequate resources to support teacher time away from the classroom, competing 
demands on teacher time, and school cultures that do not easily align with the needs of sustained, 
school-embedded, collegial work with colleagues. One of the goals of this paper is to examine 
the question of whether teacher learning communities in support of assessment for learning are 
still effective and worth pursuing, given these challenges. 

Given the steep institutional challenge associated with mounting teacher learning 
communities, it is important to say that we are not endorsing teacher learning communities as a 
one-size-fits-all solution for all teacher learning. Rather, we are endorsing a more flexible 
concept, one of matching the nature of the content to be learned with learning processes that are 
appropriate for that particular content. Where the content to be learned draws on the kinds of 
complex cognition and behaviors that are typical of experts, then we would argue that teacher 
learning communities provide a suitable, and perhaps a necessary, learning modality. Other kinds 
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of teacher learning, however, are probably best dealt with in other ways. For example, if the goal 
is to boost teachers’ subject matter knowledge, then learning vehicles that put teachers in close 
contact with expert sources (e.g., professors, texts) may be more efficient than exploring that 
subject matter with colleagues who are not experts in that realm. If the knowledge to be learned 
is procedural and highly standardized (for example, learning to use new grading and record- 
keeping software), then a workshop learning experience will be faster, cheaper, and more likely 
to result in uniform compliance. 

Professional Development in Assessment for Learning and Teacher Learning Communities 
Our Starting Point: Assessment for Learning 

Like most professional development programs, ours began by developing and trying out 
workshops in which we focused on the thing we wanted teachers to do. In our case, this meant 
we piloted, refined, and repiloted multiple versions of workshops focused on assessment for 
learning. Through the development process, we became more familiar with this content, to the 
extent that we were able to move away from a telling format in which we spent a great deal of 
time lecturing teachers about what it is and how to do it, toward a more interactive format in 
which we employed and modeled the very learning principles upon which assessment for 
learning is based. 

In the first year of our development process, the amount of content we felt we needed to 
cover grew to fill three full days. Later, we came to see that there was a limit to what teachers 
could effectively take in without opportunities for practice, reflection, and adjustment, so we 
shortened the workshop to two days. Sometimes we delivered the workshop days consecutively, 
and sometimes we spread them out, asking teachers to try out techniques in between. 
Recognizing the need to move the knowledge that was in the development team’s head to a more 
scalable format, we codified the workshop content in PowerPoint slides supported by highly 
detailed leader notes that include discussion points, timing indicators, and all needed materials. 
This resource is a valuable tool for trainers and district staff learning about the program. 

Two Content Strands: Assessment for Learning and Establishing and Sustaining Teacher 
Learning Communities 

Experience in the first districts we worked in taught us about the importance of 
institutionalizing ongoing support for teacher learning. This experience led us to understand that 



10 




we needed to explicitly address two distinct content strands in our professional development: 
assessment for learning and teacher learning communities. It’s interesting to note the staggered 
pace of our own learning about the need to attend explicitly to both areas. At first, we included 
about 20 minutes (within a 3-day workshop focused on assessment for learning) about the need 
for ongoing support for teachers in the form of teacher learning communities. This amount of 
time proved to be ineffective: we saw first hand that simply telling schools to set up teacher 
learning communities was not sufficient, just as simply telling teachers how to do assessment for 
learning was not effective. 

We then began building a 1-hour segment on teacher learning communities into the initial 
teacher workshop and created a separate workshop focused entirely on the why’s and how’s of 
teacher learning communities, directed at those charged with leading them and the school leaders 
who are charged with making way for them. This addition proved to be more effective, but still 
not sufficient. As it turns out, sustaining a teacher learning community has its own complexity 
and context-driven peculiarities, much like assessment for learning. In most schools, there 
simply isn’t enough existing know-how and cultural readiness to sustain these communities, 
even if school leaders get them off the ground. 

This knowledge eventually led us to understand that learning to do teacher learning 
communities required the development of expertise and thus would benefit from opportunities 
for learning, practice, reflection, and adjustment, just like teachers need to learn how to do 
assessment for learning. In both content areas, we were asking teachers and other school 
personnel to change deeply-held beliefs and practices, and we were challenging long-standing 
institutional roles for teachers and students. Just as an implicit classroom contract exists, an 
implicit school contract also exists, and that contract has not, historically, supported teachers to 
be learners. We began looking for ways to provide ongoing support to develop the expertise of 
teacher learning community leaders and the school leaders who must help them carve out space 
and time. 

Two Learning Phases: Initial Exposure/Motivation and Ongoing Learning 

We came to see that both content areas — assessment for learning and teacher learning 
communities — can be effectively addressed in two distinct phases: (a) initial exposure and 
motivation, and (b) ongoing learning, practice, reflection, and adjustment. In Phase I, the teacher 
or school leader is exposed to an overview and basic information about the topic through a 
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workshop or seminar presented within a motivational framework so that he or she can see the 
advantage of making a longer-term commitment to changing his or her practice. To motivate 
assessment for learning, we generally rely on compelling research that shows the student 
learning gains that can be obtained by becoming expert at it. For teacher learning communities, 
we cite both research and teachers’ own experiences with the limited effects of one-off 
workshops and then make a logical argument for sustained, collegial learning. 

Without Phase 1, most teachers would not know where to begin or even see that they 
needed to begin. But in fact, it is in Phase 2 that the learning has the potential to actually change 
teaching, learning, teachers, and schools. Phase 2 is analogous to Nonaka and Takeuchi’s (1995) 
learning by doing stage, the stage where the knowledge learned at an explicit level is translated 
into tacit knowledge that is accessible and applicable in practice in increasingly transparent 
ways. Opportunities for learning the next thing to be learned, practicing it in real settings, 
reflecting on how and why it worked or didn’t work, and adjusting and integrating the new 
learning have to be structured. Otherwise, the pace of teaching and daily life in schools do not 
naturally allow teachers and school leaders to develop expertise in complex interventions like 
assessment for learning or teacher learning communities. 

Structuring Ongoing Learning About Assessment for Learning 

At the start of the second year of our development process, we attended to structures for 
supporting ongoing learning about assessment for learning by creating modules for use by 
facilitators of teacher learning communities. These modules, comprised of directions and 
materials for 105 minutes of group study of assessment for learning, represent one of strongest 
attempts to scaffold in enough content to maintain fidelity, one of the challenges of using teacher 
learning communities to scale up any intervention. Recognizing that no assessment for learning 
experts exist in most teacher learning communities, each module provides explicit guidance for 
the conduct of a monthly teacher learning community meeting. Each module contains an agenda, 
detailed leader notes with guidance for timing and discussion points, plus informational and 
activity handouts to be photocopied for the use of participants. 

Every module begins and ends the same way, with what we have come to call the 
bookend activities. These activities are designed to provide the supportive accountability that 
teachers need to take the risk of making changes in their teaching practice. 
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• In order to model and gain the benefit of the assessment for learning strategy called 
clarifying and sharing learning intentions, each module begins and ends with a clear 
statement of the meeting’s learning intentions. 

• Every module then moves to the How’s It Going?, a segment that allows time for 
every teacher to report on and ask for feedback and help on their most recent 
experiences trying out assessment for learning techniques. Knowing that they will be 
expected to report on their most recent efforts was shown in (Paul Black, Harrison, 
Lee, Marshall, & Wiliam, 2003) to be a helpful, even necessary, spur to action. 

• Near the close of each module, a segment is devoted to personal action planning, a 
time for teachers to describe on paper their next steps in trying out and refining 
assessment for learning techniques in their own classrooms. The Personal Action 
Planning segment includes time to make arrangements to exchange observation time 
in colleagues’ classrooms or to collaborate with colleagues in other ways, perhaps to 
generate hinge -point questions for key concepts or to practice writing effective 
formative comments on student work. The expectation conveyed in this segment is 
that between meetings, teachers are to try out assessment for learning techniques in 
their classrooms. With this kind of between-meeting effort and the support of 
colleagues, teachers can gain progressively more skill and insight into how to 
improve student learning through assessment for learning. 

• Each module ends with a quick look back at the learning intentions, in which the 
group as a whole decides whether these were achieved, and if not, plans how to 
redress that problem. 

Our goal in repeating these particular opening and closing activities in every meeting is to 
create a climate and expectation of both support and accountability. By emphasizing these two 
concepts together, we hope to convey that ongoing teacher learning is worthy and necessary, that 
teachers are expected to work on improving their practice on an ongoing basis, and that they will 
be supported to do so. We believe that accountability is an important and useful tool in any 
organization. However, many teachers feel alienated from the concept of accountability, due to 
pervasive, test-heavy accountability measures that are often out of balance with capacity- 
building measures. To recapture the concept of accountability and put it in service of improving 
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teacher effectiveness, we have made the capacity-building component explicit in the way we 
structure teacher learning communities. 

In addition to the repeated bookend activities, each module also includes a teacher 
learning activity that is designed to deepen teachers’ knowledge of a particular assessment for 
learning strategy and introduce one or more associated techniques illustrated by stories of how 
real teachers have made this strategy come alive in their classrooms. As a general rule, each 
module addresses only one of the five strategies or the big idea of Keeping Learning on Track to 
keep a clear focus for the meeting. This information is usually embedded in group activities that 
require teachers to reflect on their current practices and figure out how they might adapt these 
latest techniques to their own classrooms. 

To date we have fully developed, pilot-tested, and refined eight modules, one to support 
the first meeting of a teacher learning community, and the remainder addressing the five 
strategies and one big idea of assessment for learning. Eight more are under development; these 
will get into the more challenging applications of assessment for learning (e.g., grading policies) 
and the underlying themes of assessment for learning (e.g., implications of changing the 
prevailing classroom contract between teachers and students). Our goal is to provide enough 
modules to support a rigorous 2-year course of study for a teacher learning community. Our hope 
is that after 2 years, participants will have adopted two habits of mind: (a) a student-learning 
focus will have replaced a focus on teaching as the sine qua non of teaching, and (b) teachers 
have a sense of themselves as a life-long researcher of their own craft. 

Structuring Ongoing Learning About Establishing and Sustaining Teacher Learning 
Communities 

The development, pilot testing, and refinement of the assessment for learning modules 
goes a long way toward ensuring that when teacher learning communities meet they maintain a 
strong and faithful focus on helping teachers adopt and improve the implementation of 
assessment for learning strategies and techniques. We thought that these modules might provide 
an adequate level of support for establishing and maintaining effective teacher learning 
communities. However, we learned that in most districts, the modules by themselves are not 
sufficient, because their availability does not ensure that the teacher learning communities will, 
in fact, meet. As we mentioned earlier, numerous structural and cultural barriers must be 
overcome to establish and nurture a teaching learning community. 
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The single greatest problem is the lack of time within regular school hours when teachers 
can meet to discuss teaching and learning or observe each other’s classrooms. Without this time 
during the regular, paid day, teacher learning communities can never hope to be attractive to the 
vast majority of teachers. Teacher learning community leaders and school leaders have to work 
together to communicate and demonstrate that the teacher learning community is a priority, and 
to do this, they need two things. First, they need some level of knowledge of the research base 
supporting teacher learning communities, for leverage when arguing for time and resources. 
Much of this knowledge base exists in an explicit form that can be conveyed in the initial 
exposure workshop, backed up by printed reference materials. Second, what learning community 
leaders need on top of that are ongoing, structured opportunities for new learning, practice, 
reflection, and adjustment. 

In the current school year, we have finally had the time and opportunity to begin 
development of these structures. In one district’s implementation (described below), a monthly, 
day-long workshop is required for all the teacher learning community leaders in the district 
(roughly 40 in this ambitious implementation). At this monthly workshop, the teacher learning 
community leaders function in two roles successively: first, as teachers learning about applying 
assessment for learning in their own classrooms, and second, as advocates and facilitators for 
teacher learning communities. For each role, time is set aside to reflect on successes and 
challenges, and to plan for next steps. The general framework employs the same principle of 
supportive accountability that is used in the modules. In their roles as leaders of teacher learning 
communities, the teacher-leaders are provided with a School Action Planning protocol. They are 
also given time to work together as they digest the content and processes of the upcoming 
assessment for learning module. 

A Comparison of Three Delivery Models 

We began working with schools and districts on early versions of our assessment for 
learning professional development program (before it was even called Keeping Learning on 
Track), in the 2003-2004 school year. In the 2 successive school years, we have worked with 30 
districts and hundreds of schools. Some districts have been quite large, with about 150 schools, 
whereas others have been quite small, with only 1 to 5 schools. We have worked in affluent 
suburban districts, and impoverished urban districts, with teachers at every grade level and 
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subject. Along the way, our understanding of what was required to support teachers to make 
significant, sustained changes in their practice evolved to the elements described above. 

In the rest of this paper, we will focus on implementations delivered in two districts and 
one consortium of districts, which collectively illustrate the stages of development of our own 
thinking and how the design of the program evolved over time. General characteristics of the 
districts and the consortium are summarized in Table 1, which uses pseudonyms for each locale 
and omits the state of the large urban district to reduce the probability of recognition. 

In each of these implementations, the terms, nature, and frequency of our engagement 
with schools and teachers varied for three reasons: 

• We were at different stages of our development process when we began working in 
the different districts. In the early years, we had not yet developed the notion or the 
content of the teacher learning community modules. It was only in the recent period 
that we began developing the ongoing support systems for teacher learning 
community leaders and school leaders. 

• We were pursuing different research goals in different districts at different times. Our 
early focus was on teaching teachers about assessment for learning, but as time went 
on we became equally interested in teaching teachers and other school personnel 
about teacher learning communities. We also were willing to provide nonscalable 
professional development in one close-at-hand district in exchange for greater access 
to teachers for interviews and observation. 

• We became more skillful over time about how to negotiate firm commitments from 
schools and districts regarding their ongoing support for assessment for learning, 
teacher learning communities, and providing us with data. 

• These varying circumstances led to different implementations, or delivery models, in 
the three locations. Though the implementations were not deliberately designed to 
systematically test all possible variations, they represent a range of delivery models 
that afford insight into factors affecting impact on teaching practices and scalability. 
In the next sections of the paper, we examine these differences with regard to the 
components that were implemented in each locale and their relative impact and 
scalability. 
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Table 1 



Characteristics of Districts and Their Implementations of Keeping Learning on Track 





Gateway district 


Consortium 


Corolla district 


Type 


Single district 


18 Districts 


Single district 


Location 


New Jersey 


New Jersey 


Midwest 


District size 


Small 


Small to medium 


Large 


Population served 


Middle class 


Middle to upper 
middle class 


Poor and 
working class 


Date research partnership 
began 


Fall 2004 


Summer 2005 


Fall 2005 


Duration of partnership 


2 years 


1 year 


1 year 


District support for initiative 


Moderate 


Weak 


Strong 


Level of teacher contact 
with AfL experts 


High 


Low 


Low 


Initial training in AfL 


Strong 


Strong 


Strong 


Initial training in TLCs 


None 


Moderate 


Strong 


Initial training in AfL/TLCs 
provided for school leaders 


None 


Weak 


Strong 


AfL modules available 
and used? 


Not available 
in Year 1, led 
by experts in 
Year 2 


Moderate use, 
led by teachers 


Strong use, 
led by teachers 


Ongoing support for TLC 
leaders/ school leaders 


None 


None 


Developing 



Note. AfL = assessment for learning; TLC = teacher learning community. 



Gateway 

Lyon, Wylie, and Goe (2006) described the Gateway implementation in detail. Here, we 
focus on high-level implementation traits. Gateway district is a small suburban district and 
relatively forward thinking with regard to teacher professional development. The district 
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leadership initiated the relationship with our development team after reading about the British 
assessment for learning work, upon which Keeping Learning on Track is based. 

Of all the implementations, the Gateway district received the highest level of expert 
assistance in assessment for learning during both the initial and ongoing learning phases. Dylan 
Wiliam, an internationally recognized expert in assessment for learning, provided multiple 
workshops and after-school seminars on assessment for learning to Gateway teachers over the 
course of 2 years, in exchange for ready access to teacher feedback, largely through journal 
entries they were required to make as a condition of the arrangement. 

In the first year, some informal teacher learning activities occurred within the time set 
aside for grade-level teams, with tacit support from the administration for talking about 
assessment for learning. However, during this first year, no time was explicitly set aside for 
teacher learning communities devoted to assessment for learning, and no modules were available 
to help guide such meetings. This gap left participants to figure out for themselves how best to 
support one another. To some extent, they based their processes on the book. Assessment for 
Learning: Putting it into Practice (Paul Black et al., 2003), which documented the British 
assessment for learning work upon which Keeping Learning on Track is based. But for the most 
part, the closest they came to a true teacher learning community experience was in their monthly 
seminars with Dylan, when he led teachers through cycles of reporting on their assessment for 
learning efforts and gave supportive feedback. In effect, he became a de facto leader of a larger- 
than-normal, district- wide teacher learning community. 

In the second year of the program in Gateway, a large number of second generation 
teachers joined because they had heard good things about assessment for learning from their first 
generation colleagues. These teachers attended a second series of workshops and seminars 
conducted by Dylan, and a few formed a teacher learning community under the leadership of a 
particularly dedicated first generation teacher. However, this group found it extremely difficult to 
find a common meeting time, so they have only been able to meet two times over the course of 
the current school year. Except for the teachers in this one group, the second generation teachers 
were supported entirely by Dylan in the monthly after-school sessions. The second year of the 
program in Gateway coincided with the development of the first modules, and some of these 
were pilot tested by Dylan in his monthly after-school seminars in Gateway. 



18 




In summary, the Gateway implementation relied heavily on an outside expert for both the 
initial exposure and the ongoing learning opportunities, and little formal capacity building was 
pointed at the development of local expertise through teacher learning communities. 

In terms of the impact of the intervention, many teachers have become assessment for 
learning devotees, documenting in their journals sincere and sustained efforts at incorporating a 
wide array of strategies and techniques that seem appropriate for their teaching situations. 
Classroom observations suggest that about half of those teachers in the highest exposure group — 
the teachers in the one formal teacher learning community — are actually becoming quite 
proficient at assessment for learning. However, not all are on track — some talk a good game 
regarding assessment for learning, but they have not yet succeeded at putting it into practice. 
Nevertheless, a clear district- wide bent toward assessment for learning appears to have staying 
power. This development may be attributable to the high touch, high fidelity immersion that the 
district got, learning about assessment for learning straight from the horse’s mouth. If district 
support for assessment for learning continues as we expect it will, we have reason to hope that 
more teachers will improve their practice as this year’s work unfolds. 

Despite the moderately promising results in the Gateway district, one obvious limitation 
to this delivery model is apparent: Because of the dependence on expert leadership at almost 
every turn, the model is in no way scalable. 

The Consortium 

The Consortium’s implementation represents one of our more ambitious efforts at scaling 
up, in which we consciously attempted to establish teacher learning communities in 18 small to 
mid-sized districts in New Jersey. The 18 districts were members of a preexisting Consortium 
with a stated interest in performance assessment. Seeing some overlaps between the theory and 
methods of performance assessment and assessment for learning, the convenor of the network 
invited Dylan to conduct a 5-day summer workshop on assessment for learning. Seeing this as an 
opportunity to develop and test out our support systems for teacher learning communities, Dylan 
pushed back on the Consortium, requesting that the districts commit to sustained professional 
development in the form of teacher learning communities, and that the 5 days be split between a 
3-day training in assessment for learning for teachers and a 2-day training for leaders of teacher 
learning communities and their principals. The first day of the learning community training 
would take place on Day One of the 5-day run; the second day would follow at the end of the 
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assessment for learning workshop. We expected roughly 30 leaders to show up for Day One, 
roughly three per district, including principals or their surrogates. 

The Consortium opportunity was the spur we needed to develop and provide initial 
exposure and motivation in both assessment for learning and teacher learning communities. 
However, upon arriving for the first day of training with the leaders, we were disappointed to 
learn that communications about the requirements for district participation had gotten quite 
garbled and diluted. Teachers and school leaders who we thought had volunteered to lead 
ongoing teacher learning communities had no idea that their obligations extended beyond the 
summer workshop. Only a few districts were represented by central office staff empowered to 
make changes in the district’s professional development plans for the upcoming school year. The 
leaders who were present were as surprised as we were about the miscommunication. Once we 
all got over the shock, we made several adaptations to the first day of training, spending 
considerably more time on the motivation for teacher learning communities than we had 
originally planned — since we had to convince them it was worth extending themselves past their 
original commitments. 

Despite the rocky start for the teacher learning community portion of the training, the 
transmission of explicit knowledge concerning assessment for learning went very well. Over 100 
teachers and school leaders remained attentive, responsive, and engaged over 3 long workshop 
days. On the last day of the assessment for learning workshop, teachers completed their first 
personal action plans. Most were ambitious yet reasonably well focused on plausible first steps 
in assessment for learning. Evaluations for the session echoed those we had received at earlier 
workshops: Almost all were wildly positive about assessment for learning and the activities of 
the workshop. 

The work with the leaders of teacher learning communities was not so seamless. Despite 
the confusion about their ongoing obligations, most of the leaders agreed to continue into their 
second day of leader training and then return to their districts to arrange whatever they could 
personally support for the coming year. A few taught at schools with preexisting professional 
learning communities, which they hoped to convince to focus on assessment for learning. Most, 
however, knew they would have to expend a fair amount of time, energy, and personal capital 
convincing colleagues and administrators that this endeavor was worthwhile and deserved their 
involvement and support. 
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We had developed early versions of the first two modules, which we supplied to the 
leaders, and these were appreciated by the leaders. But their greatest concerns centered on how 
they were going to muster the permissions, resources, and time for regular 2-hour meetings with 
colleagues and how they were going to convince their colleagues to participate, especially since 
few districts were setting aside time within the school day or offering to pay stipends for after- 
school participation. 

On the fly, we adapted many of the planned workshop segments for teacher learning 
community leaders into structured brainstorming sessions, employing force field and strengths- 
weaknesses-opportunities-threats (SWOT) analysis techniques to help them focus on the likely 
internal and external constraints and affordances for instituting viable learning communities and 
to leverage the good ideas to be found within a group of dedicated educators. We quoted a 
Chinese proverb. Three heads make a genius, and explained that they were getting a chance to 
experience the kind of group support we hoped they would foster in their teacher learning 
communities. The leaders told us that this was useful to them, but they were still very worried 
about being expected to take charge of such a radical reshaping of professional development in 
their schools. 

We knew at the end of the training that the chances of many of these districts establishing 
ongoing teacher learning communities were slim. We wished we had a ready mechanism for 
ongoing support for these districts. Unfortunately, we were stymied by three limitations. First, 
the districts in the Consortium were only loosely coupled, and the Consortium did not have a 
strong, centralized communications network. This meant that any ongoing work would largely 
have to be facilitated between our research group and each individual district. Second, we had 
only the beginnings of a plan for how to support ongoing learning focused on teacher learning 
communities — it seemed we were learning about this right along with the districts. Our rather 
simplistic plan going into the summer workshop was to provide them with modules throughout 
the year and perhaps hold a midyear miniconference for leaders of teacher learning communities. 
Third, we were already scheduled to take on our next scalability challenge: working in a large, 
urban district, located hundreds of miles away. We simply didn’t have the time to devote to 
salvage work in the Consortium districts. 

We cannot gauge the effectiveness of the professional development we provided to the 
Consortium in terms of its impact on actual teaching practices because we have not had the 
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resources to interview or observe teachers. In recent months, however, we have invested 
resources in following up with at least one teacher learning community leader for each district. 
We are phoning, emailing, and conducting in-person interviews to ascertain the presence and 
viability of learning communities in each district. For those that never took shape or died on the 
vine, we are trying to find out the specific barriers they encountered. 

Of the 18 districts that sent representatives to the summer workshops, two dropped out by 
October, stating that they “didn’t know it was a year-long process.” Four of the remaining 16 
districts have responded to our first round of inquiries. Not surprisingly, these four are districts in 
which some form of teacher learning community has gotten off the ground; they have a story 
they want to tell. Three of these seem to be making use of the modules we send them. The fourth 
is a district where teachers are operating with no contract and after-school professional 
development is out of the question. However, they are attempting to discuss assessment for 
learning strategies and techniques in their regularly scheduled faculty meetings and hope to 
formali z e the work once a contract is signed. We do not yet know what has happened in the 12 
districts where we cannot get a response to our repeated calls and emails. We suspect that the 
news is not good in these locales; otherwise we think they would be eager to tell us their story 
and ask for more resources, which is what the four responders have done. 

We would have to say that the Consortium delivery model was underconceptualized and 
weakly executed on both our end and the district’s end. Given the likely outcome for the teachers 
in the 12 districts we cannot reach and the entire lack of ongoing support in the two that dropped 
out at the start of the school year, it is highly unlikely that teacher practice has changed in 
significant ways for most teachers who were exposed to the intervention. Thus, we would have 
to classify this effort as largely ineffective and prescalable. 

However, the effort was not a waste: We learned that we needed to provide ongoing 
support for teacher learning communities and we gained a real-world bad example we can point 
to regarding the need for substantial up-front buy-in on the part of the district or school. We took 
these lessons to our next implementation, in Corolla. 

Corolla 

Corolla is a large, midwest school district with a declining tax base, more than 100 
schools (many in a poor state of repair), a student population that comes largely from working 
class or poor households, and a long-standing record of weak academic performance. Under 
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strong leadership in the late nineties and the early part of the current decade, the district made 
some turnarounds in attendance, graduation, and other indicators of performance, to some extent 
renewing hope for some staff, parents, and students. However, recent funding cutbacks at the 
state level and the defeat of a local bond issue led to sizable and demoralizing staffing cuts, 
increases in class size, building closures, and the eventual resignation of the CEO who had 
overseen many of these improvements. 

This challenged district provides the backdrop to our most ambitious scaling effort to 
date: bringing teacher learning communities focused on assessment for learning to life in 10 Title 
I schools, each school in its third or fourth year of failing to make Adequate Yearly Progress, as 
defined by the No Child Left Behind Act. To add to the scaling challenge, the district is hundreds 
of miles from ETS, which meant that we would not be able to provide any form of regular hand 
holding. Instead, after the initial exposure phase, we would have to develop capacity at the 
district level to support the teacher learning communities and help them keep a strong, clear 
focus on assessment for learning. 

We were encouraged that we would be able to accomplish this goal because our 
relationship with Corolla originated in a connection with a very capable, highly placed central 
office administrator who understood the principles of assessment for learning and its potential. 
She was prepared to prioritize this work in every way she could. Without such an informed, 
committed advocate, we would not have taken on this challenge — having learned the previous 
summer just how critical such internal buy-in is. 

Before heading off to conduct the initial rounds of training for teachers, learning 
community leaders, school leaders, and district leaders, we redeveloped and augmented the 
initial training sessions, making changes based on the gaps we identified in the Consortium 
summer workshops. We also worked much more closely with the district leadership in Corolla, 
to ensure that the teachers and their schools had a clear understanding of the expectations 
surrounding ongoing participation and support throughout the year. These expectations were 
spelled out in detail in a seven-page memorandum of understanding, signed by the CEO of the 
district and ETS, which captured the roles and responsibilities of both parties. The level of 
support the initiative sees from the district is substantial, and we believe that the process of 
hammering out this written document is an important reason why. One of the requirements of the 
memorandum of understanding was that the district must share student achievement data for this 
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year and last, for the students of teachers in the program, and for teachers in a matched sample of 
classes. This data will allow us to test whether the intensity of a teacher’s exposure to the 
intervention correlates with his or her students’ average learning gains. 

Our professional development effort in Corolla began with a 1-day workshop designed 
for central office-based administrators who were charged with supporting the initiative in the 10 
schools. The workshop was intended to motivate these leaders by presenting the research on both 
assessment for learning and the research on growing teacher expertise through sustained, school- 
embedded teacher learning communities. Unfortunately, the workshop was inadvertently 
scheduled on a day when principals were told to be at their schools for a last-minute district- wide 
head-count. Only the central office-based staff (about 15 people) participated. However, the 
workshop was well received, and most of these same staff decided to attend the next two 
workshops, so that they could soak up all they could about assessment for learning and teacher 
learning communities. 

The scheduling conflict we experienced on our first day in Corolla is not uncommon in 
that district or in other large urban districts that are battling histories of low performance. Such 
districts are complex organizations that teeter under the weight of conflicting programs and 
priorities, all brought in for the noble purpose of improving achievement or at least minimizing 
system failures. Throughout our year in the district, we have watched as our internal champion 
struggled to push the initiative back to the top of the priority lists of high, middle, and lower 
level leaders. She often succeeds, and because of her constant advocacy, teacher learning 
communities are up and running in all 10 schools and the strategies and techniques of assessment 
for learning can be seen in many classrooms in these schools. On the other hand, a number of 
disappointments have also occurred along the way, in particular in getting building principals on 
board in any significant way. In one school among the ten the principal is deeply involved; in the 
rest, it is unclear what the principal knows about the initiative; and in some schools, it is not 
uncommon for the principal to schedule other activities that directly conflict with the learning 
community meetings. 

Following the workshop for district and school building leaders, we conducted a 2-day 
initial workshop for teacher learning community leaders, with the first day focusing on 
assessment for learning, and the second day focusing on their roles as leaders of learning 
communities. Each of the 10 schools sent two to seven teachers to this workshop (though they 
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were all told to send seven by the CEO). This group of roughly 40 teacher- leaders has continued 
to meet once a month, usually under the leadership of the internal district champion and twice 
under our leadership. The group has made good strides in terms of the individual members’ 
understanding of assessment for learning and in terms of its grasp of the nuts and bolts of leading 
sustained school-based learning communities for teachers. The monthly meetings follow the 
protocols described earlier, attending to both the teacher and leader roles. 

Following the workshop for teacher learning community leaders, we provided a series of 
three 1-day workshops for teachers in the 10 schools. Every teacher in the 10 schools was invited 
to attend one of these workshops. We were prepared for a total of 200 to 300 teachers to attend 
one or another workshop, but a lack of substitute teachers made it difficult for many to attend. 
The funds for substitutes had been obtained and made available to the 10 schools for this 
purpose; there simply were not enough substitutes available. This meant that many teachers were 
forced to ask their colleagues to cover their classes for them or to skip coming altogether. Clear, 
written descriptions of the purpose and focus of the workshops were provided to each school’s 
principal, but these were not always relayed to the teachers, so this too cut into attendance. 
Nevertheless, we eventually attracted about 120 teachers to a workshop. 

Once there, the teachers responded very positively, similar to teachers in earlier 
workshops. They were, however, skeptical of the ongoing support that was promised and 
immediately worried that this was just the latest reform du jour, to be replaced by the next 
education fad taken up by the district. We worked with district staff to assure them otherwise, but 
truthfully, we did not know what support would materialize as the months went by. 

As it turns out, the support at the district and school level has been reasonably good. At 
the district level, day-to-day support comes mostly from staff who report directly to our 
champion, whereas staff who report to others seem to have let the initiative slide from their 
priority lists. The district CEO has made her support known by making a brief address at one of 
the workshops and by sponsoring a day-long seminar on the initiative for all principals and 
building-level union representatives in the district. Funds have been found to pay a stipend to 
teachers who stay after school to participate in teacher learning communities, and funds are 
available for substitutes so that teachers can visit each other’s classrooms — however, such visits 
are hampered by a severe shortage of substitutes. Funding has been set aside to deepen and 
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spread the work in the coming school year, a sign that this effort might be more than just the 
latest fad. 

At the school level, teacher learning communities have met with varying degrees of 
regularity in all 10 schools. Some have as many as 18 members (far more than we advised, but 
they did not want to break into smaller groups, this being their “only chance to get to know one 
another!” as one of them put it) and some have as few as three — all three being teachers who 
were originally trained as teacher learning community leaders. In some schools, recruitment has 
extended beyond the pool of teachers who attended the initial workshop. This extension means 
that some teachers who have never been exposed to an ETS trainer are participating in a learning 
community — at this point, this is the furthest remove at which we have operated in our quest for 
scalability. As of March, most learning communities have worked through three or four modules, 
which is a reasonable number, since the training finished in October, the first meetings were held 
in November, and some meetings were canceled due to testing and holiday conflicts. 

We have trained and placed local researchers in most learning community meetings held 
since January and have found that the leaders are diligently and faithfully trying to cover the 
material. Since we have discovered this tendency to emphasize coverage over understanding and 
community building, we have used our two sessions with the leaders to emphasize the latter. We 
have begun to see more attention to the sharing of their experiences with assessment for learning 
and some fledgling attempts at teachers holding each other accountable for changing and 
improving their practice with regard to the big idea of assessment for learning. This 
accountability often shows itself in the form of a prompt we have given the leaders: What’s 
formative about that? We encourage them to follow up with this question or a related one to any 
incomplete or vague description of practices billed as being about assessment for learning. We 
find that supporting teachers to hold each other accountable is one of the trickiest issues we 
face — a culture of polite, serial listening prevails, with many teachers believing that it is not their 
place to challenge their colleagues. The isolation of years of teaching behind closed doors has 
not prepared teachers to share their struggles, ask for help, or provide critical analysis. 

We have also begun to observe teachers in their classrooms to assess the frequency and 
skill with which they are trying out assessment for learning techniques. As in the Gateway 
district, we find that some teachers have been able to usefully integrate assessment for learning 
techniques into their practice, drawing out information about student learning which they then 



26 




use to modify their instruction. We have noted that in many classrooms, the selected techniques 
cluster around those who require physical tools, such as the Popsicle sticks used to randomly 
select a single student to answer questions, and the ABCD cards and white boards of whole-class 
response systems. Nothing is wrong with any of these choices in themselves, but the frequency 
with which teachers select these for the focus of their efforts suggests that the teachers’ 
understanding of the breadth of assessment for learning is still at the concrete stage where 
manipulatives help to make the technique real. The next step will be for these teachers to show 
that they can make flexible use of the information these techniques evoke, which might lead 
them to some of the less obvious techniques associated with learning intentions, feedback, and 
activating students and peers as instructional resources for one another. 

In terms of impact on teacher practices, then, we are seeing real efforts at change on the 
part of most teachers we observe, and some of these teachers are reaching proficiency with some 
techniques. A few seem to really grasp the big idea. Overall, we would say that the impact on 
teaching practices is moderate. In terms of scalability, the Corolla implementation has succeeded 
in reaching more teachers in more schools than any other delivery model we have tried. Also, the 
potential exists for increasing the scale of the Corolla implementation model, as other schools in 
the district are thinking about joining the initiative without direct support from us, and additional 
teachers within the starter schools are joining the existing learning communities. 

Discussion 

At the outset of this paper, we outlined the goal of building models of professional 
development that are both effective and scalable. We explained how we arrived at the notion of 
school-based teacher learning communities as a vehicle for accomplishing this goal, developing a 
theoretical and empirically-based argument for making this approach a central component in our 
general model of professional development. We then described the process by which we realized 
that the content of assessment for learning was not the only content that we needed to convey 
explicitly to teachers and school leaders. This process was necessary because the establishment 
and maintenance of teacher learning communities runs contrary to multiple prevailing conditions 
in schools, especially with regard to providing time and structures within the school day for 
supporting teachers as learners. Finally, we described how we learned from practical experience 
and the application of general learning theory that we had to attend to two distinct phases of 
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professional development: a) initial exposure and motivation and b) ongoing learning — and that 
we needed to attend to both of these phases for each of our featured content areas. 

To aid in conceptualizing this body of thought, we offer Figure 1, which places the two 
content areas in relation to the two learning phases described above. Crossing these two 
dimensions yields a two-by-two matrix. In theory, an effective and scalable program of 
professional development would require attention to each quadrant in the matrix. 





Content Areas 


Assessment for Learning 


Teacher Learning 
Communities 


Learning 

Phases 


Initial Exposure and 
Motivation 


Quadrant A 


Quadrant C 


Ongoing Learning, 
Practice, Reflection, 
and Adjustment 


Quadrant B 


Quadrant D 



Figure 1. Keeping Learning on Track: Content areas and learning phases. 



At our earliest and lowest levels of implementation, we focused exclusively on initial 
exposure and motivation with regard to assessment for learning (Quadrant A), not unlike most 
currently available professional development. As our understanding evolved through praxis and 
research, we began to address teachers’ ongoing need for support through the vehicle of teacher 
learning communities (Quadrant B). As our early implementations failed to demonstrate both 
effectiveness and scalability, we realized the need to provide explicit training to learning 
community leaders and school leaders to help them get their learning communities off the ground 
(Quadrant C). Finally, we came to see the need for ongoing support for learning by doing in the 
implementation of learning communities (Quadrant D). 

Our research and development process and the terms of the arrangements made with our 
district research partners led us to direct different amounts of attention to the four quadrants 
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named above in Figure 1 . These differences are summarized in Figure 2, in which each 
implementation is listed in those quadrants for which there was deliberate, structured attention. 





Content Areas 


Assessment for Learning 


Teacher Learning 
Communities 


Learning 

Phases 


Initial Exposure and 
Motivation 


Gateway Consortium 
Corolla 


Consortium Corolla 


Ongoing Learning, 
Practice, Reflection, 
and Adjustment 


Gateway-Corolla 


Corolla 



Figure 2. The delivery models’ relative emphasis on learning phases and content areas. 



In only one of the three implementations did we deliberately and systematically provide 
support for both learning phases in both content areas: Corolla. In Corolla we collected more 
systematic data regarding the implementation, and teachers in Corolla are responding to the 
intervention by making changes consistent with assessment for learning in their teaching 
practices (Wylie, Lyon, Ellsworth, & Martinez, 2007). These results are particularly noteworthy 
in light of the challenging working conditions in Corolla. 

We are encouraged by these results, but we are still not as far along with regard to 
scalability as we need to be, if we want to build a professional development model that can reach 
all the schools in the country that need it. The trend line across our development path is 
definitely heading toward increased scalability. In Gateway, we had moderate to strong take-up 
of assessment for learning among the participating teachers, but this was to be expected, given 
the high-powered, expert resources that were brought to bear — one of the leading lights on 
assessment for learning introduced teachers to assessment for learning and personally guided 
them through more than a year’s worth of learning. In the Consortium, we attempted to move to 
scale, but didn’t adequately conceptualize our own role or the role of the districts in bringing this 
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about. Thus, our carefully designed professional development in both assessment for learning 
and teacher learning communities was largely wasted. Finally, in Corolla, we were able to put 
the pieces together to move the intervention to scale, and we were able to reach more than 120 
teachers in 10 schools with moderate impact on teachers’ practices. 

Conclusion 

The perennial issue as we scale up further, of course, will be fidelity to the high quality 
professional development we can offer when we are able to send in the experts. How can we 
progressively tweak the content and the learning modes so that a reasonable level of impact on 
teacher practices can be maintained while reaching even more teachers in more schools? Our 
experience suggests that increasing scalability is possible, but it can’t be done overnight. Rather, 
an iterative research and development process will be needed, one that provides opportunities to 
make adjustments and then test them out, and allows us time to develop and add new resources 
for a widening base of stakeholders. 

None of the ideas in the framework we present in this paper seem especially 
revolutionary; the thinking reflects the application of basic learning theory, similar to the ideas 
laid out in How People Learn, by (Bransford, Brown, & Cocking, 1999). As we write, it strikes 
us that our application of general learning theory to teacher learning (about assessment for 
learning) and institutional learning (about sustaining teacher learning in learning communities) 
should not have taken so long or required us to learn so many lessons the hard way. We suspect, 
even though we had read Bransford et ah, Nonaka and Takeuchi (1995), and other learning 
theorists, that applying these lessons to the development of effective and scalable teacher 
professional development is itself a knowledge creation and transfer task that can only be solved 
through learning by doing. 
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Notes 



* This paper was written when Marnie Thompson was on staff at ETS. 

2 

Some educators use the term content to refer exclusively to the notion of subject-matter — what 
is being learned in the classroom (as opposed to how it is taught — pedagogy), usually 
understood as math, language arts, science, social studies, arts, etc. In the professional 
development world, however, the term content refers to what is being learned by teachers 
undergoing professional development, which may include general pedagogy, subject-specific 
pedagogy, or straight subject matter knowledge. The content of professional development at 
the center of this paper is focused on general pedagogy and subject- specific pedagogy, not 
traditional subject matter knowledge. 
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